Augmented reality based injection therapy

ABSTRACT

Approaches for augmented reality systems and methods for administering repeated injections are provided. An augmented reality (AR) system for administering injections includes: an AR headset comprising a display that is configured to display at least one marker superimposed on a real-world view of a subject, the at least one marker being displayed at a user-defined location on the subject. The display may be configured to continuously display the at least one marker at the location on the subject as the real-world view of the subject changes. The marker may be generated based on user input received from a user wearing the AR headset. The location on the subject may be defined by a location of a marker device relative to the subject when the user input is received. The AR headset may include a face mapping system.

BACKGROUND

The present invention generally relates to medical devices andprocedures and, more particularly, to augmented reality systems andmethods for administering repeated injections.

Subcutaneous and intramuscular administration of a botulinum toxin isused for treating various diseases and for cosmetic applications.Typically, a syringe or a needleless device is used to inject thebotulinum toxin to the dermal or subdermal target tissue. For somediseases, such as neuralgia, multiple injections of the botulinum toxincan be required over a relatively small area of the skin. Multipleinjections are carried out to achieve a desired distribution andtherapeutic diffusion of the botulinum toxin into the target area, asopposed to making only one or a few injections.

Injection therapy is commonly carried out over a number of discreteprocedures that may be separated by weeks or months. In one exemplaryscenario, a patient indicates that the previous injection treatment washighly satisfactory and requests that the provider “do what you did lasttime,” which essentially is a request to administer the injections ofthe current procedure in the exact same locations as the injections ofthe previous procedure. However, it is very difficult if not impossiblefor the provider to administer the injections in the exact samelocations as they were administered in the previous office visit. Thisis because the provider is essentially going from memory as to wherethey administered the injections in the previous visit. As a result, itis nearly impossible to precisely repeat the same injections in the samelocations from one injection treatment procedure to the next.

SUMMARY

In a first aspect of the invention, there is an augmented reality (AR)system for administering injections, comprising: an AR headsetcomprising a display that is configured to display at least one markersuperimposed on a real-world view of a subject, the at least one markerbeing displayed at a user-defined location on the subject. Inembodiments, the display is configured to continuously display the atleast one marker at the location on the subject as the real-world viewof the subject changes. In embodiments, the marker is generated based onuser input received from a user wearing the AR headset. In embodiments,the location on the subject is defined by a location of a marker devicerelative to the subject when the user input is received. The user inputmay be: a voice command, a button press on the marker device, or basedon a means to detect when the marker device comes into physical contactwith the subject.

In embodiments, data defining the at least one marker is stored incomputer memory. The data may define the location on the subject. Thedata may define at least one of: a size of the at least one marker, ashape of the at least one marker; and a color of the at least onemarker. The system may be configured to receive user input to change atleast one of: the location on the subject; a size of the at least onemarker, a shape of the at least one marker; and a color of the at leastone marker. In embodiments, the system is configured to: automaticallyidentify the subject; in response to the automatically identifying thesubject, access the data defining the at least one marker; and inresponse to the accessing the data, automatically display the at leastone marker superimposed on another real-world view of a subject, the atleast one marker being displayed at the user-defined location on thesubject.

In an embodiment, the AR headset comprises a direct view device, thedisplay is substantially transparent, the real-world view of the subjectis seen through the substantially transparent display. In anotherembodiment, the AR headset comprises an indirect view device, and thereal-world view of the subject is real-time camera imagery that isdisplayed via the display.

The AR headset may comprise a face mapping system. The face mappingsystem may comprise: an infrared emitter that projects a number of dotsin a predefined pattern onto a surface of a person's face; and aninfrared camera that photographs an image of the dots on the surface ofthe person's face. The location of the at least one marker on thesubject may be defined relative to a facial map of the subject generatedby the face mapping system.

The AR headset may comprise one of: goggles, glasses, a headband, and ahelmet.

In another aspect of the invention, there is a method of using augmentedreality (AR) for administering injections, comprising: receiving userinput defining at least one marker at a location on a subject; anddisplaying, by a display of an AR headset, the at least one markersuperimposed on a real-world view of the subject, the at least onemarker being displayed at the user-defined location on the subject. Themethod may further comprise continuously displaying the at least onemarker at the location on the subject as the real-world view of thesubject changes.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention is described in the detailed description whichfollows, in reference to the noted plurality of drawings by way ofnon-limiting examples of exemplary embodiments of the present invention.

FIGS. 1A and 1B show exemplary shapes of an AR system in accordance withaspects of the invention.

FIG. 2 shows a block diagram of an AR system in accordance with aspectsof the invention.

FIGS. 3A, 3B, 4A, 4B, 5A, 5B, 6A, 6B, 7A, and 7B illustratefunctionality of the AR system in accordance with aspects of theinvention.

FIGS. 8 and 9 show aspects of a recognition system usable with an ARsystem in accordance with aspects of the invention.

FIG. 10 shows an exemplary marker device in accordance with aspects ofthe invention.

FIG. 11 shows a flowchart of a method in accordance with aspects of theinvention.

DETAILED DESCRIPTION

The particulars shown herein are by way of example and for purposes ofillustrative discussion of the embodiments of the present invention onlyand are presented in the cause of providing what is believed to be themost useful and readily understood description of the principles andconceptual aspects of the present invention. In this regard, no attemptis made to show structural details of the present invention in moredetail than is necessary for the fundamental understanding of thepresent invention, the description taken with the drawings makingapparent to those skilled in the art how the several forms of thepresent invention may be embodied in practice.

The present invention generally relates to medical devices andprocedures and, more particularly, to augmented reality systems andmethods for administering repeated injections. Augmented reality (AR) isa live direct or indirect view of a physical, real-world environmentwhose elements are augmented by computer-generated imagery. Augmentedreality may be implemented in wearable display systems that are embodiedin a wearable headset that is arranged to display an image within ashort distance from a human eye. Such wearable headsets are sometimesreferred to as head mounted displays (HMDs). Optical components arearranged in a wearable headpiece so as to display the desired imagewithin a few centimeters of the user's eyes.

Aspects of the invention leverage an AR system in an injection therapymethodology. Implementations of the invention utilize AR in a mannerthat advantageously permits a provider, such as a physician who isadministering injection therapy to a patient, to precisely repeat thesame injections in the same locations on the patient from one injectiontreatment procedure to the next.

FIGS. 1A and 1B show examples of a wearable AR headset 50 in accordancewith aspects of the invention. The headset 50 is arranged to display animage within a short distance from a human eye of a wearer 40. Differentshapes of headset 50 may be used within the scope of the invention. Forexample, as shown in FIG. 1A the headset 50 may comprise a goggle-likeshape, and as shown in FIG. 1B the headset 50 may comprise aglasses-like shape. Other shapes may also be used for the headset 50,including but not limited to a helmet and a headband.

FIG. 2 shows a block diagram of an AR system including the headset 50 inaccordance with aspects of the invention. In embodiments, the AR systemincludes at least one computer processor 52, at least one computermemory 54, at least one visual display 56, at least one camera 58, andat least one sensor 60. These components, and the circuitry thatoperatively connects them, may be embedded in and/or attached to astructural member of the headset 50 (e.g., a glasses/goggles frame,headband, etc.), as diagrammatically shown in FIG. 1B. Similarly,although not shown, these components and the circuitry that operativelyconnects them may be embedded in and/or attached to a structural memberof the headset 50 of FIG. 1A and any other headset used inimplementations of the invention.

Processor 52 may comprise any number and suitable type of processingunit including without limitation a CPU, a GPU, and an FPGA. Memory 54may comprise any suitable type of memory, such as RAM, cache, and flashmemory, for example. Memory 54 may store, by way of example and notlimitation, an operating system, one or more application programs, otherprogram modules, and program data, which may be executed by theprocessor 52 to carry out one or more of the functions described herein.

Display 56 comprises one or more display elements that are configured todisplay a visual image within a short distance from a human eye of awearer 40. Display 56 may comprise, but is not limited to, a waveguidedisplay, a projection display, and an LCD display.

In embodiments, the camera 58 comprises at least one forward facingcamera that is configured to capture video imagery of the real-worldenvironment that is in front of the wearer 40. In a preferredembodiment, the camera 58 comprises two forward facing cameras, and in amore preferred embodiment the camera 58 comprises two forward facingstereo cameras configured to capture an approximate view (e.g., field ofview) from the wearer's left and right eyes respectfully. The two stereocameras may be located towards either side of the wearer's head on theheadpiece, and thus capture images of the scene forward of the deviceform slightly different perspectives. In combination, the stereo camerascapture a stereoscopic moving image of the real-world environment withinthe field of view of the wearer.

In embodiments, the at least one sensor 60 is configured to determine aspatial position of the headset 50 in six degrees of freedom comprising:Cartesian coordinates (x,y,z) of the headset 50 with respect to asuitable origin; and roll, pitch, and yaw (R, P, Y) of the headset 50with respect to suitable reference axes relative to the origin. The atleast one sensor 60 may comprise at least one of an accelerometer, agyro, a compass, a GPS sensor, and a proximity sensor. Any suitablesensor technology may be used, including but not limited to MEMS(microelectromechanical systems) sensors.

The electrical components of the headset 50 may be powered by a battery62 that is carried by (or part of) the headset 50. Alternatively, theheadset 50 may be physically wired to an electrical power source 66 thatprovides power to the electrical components of the headset 50.

The headset 50 may include an interface 64, which may be a wired and/orwireless communication interface by which the headset 50 communicateswith another computer device 68 and/or a marker device 70. For example,the interface 64 may comprise at least one antenna that is configuredfor wireless communication between the headset 50 and another computerdevice 68 and/or a marker device 70 via at least one of WiFi, Bluetooth,NFC, etc. Additionally, or alternatively, the interface 64 may comprisea wired link between the headset 50 and another computer device 68and/or a marker device 70.

The headset 50 may be configured as either a direct view AR device or anindirect view AR device. In the direct view configuration, the user 40wearing the headset 50 sees through a substantially transparent display56, and AR objects are rendered on the display 56 in locations thatcoincide with the real-world object that is being directly viewedthrough the display. When the headset 50 comprises a direct view device,the AR rendering may be performed using hardware and techniques such asthat described U.S. Patent Application Publication No. 2017/0053447,dated Feb. 23, 2017, the disclosure of which is incorporated byreference herein its entirety. For example, in the direct viewimplementation, the display 56 may comprise an at least one opticalcomponent that is a substantially transparent waveguide, whereby thewearer can see through it to view a real-world environment in which theyare located simultaneously with a projected AR object, thereby providingan augmented reality experience. Other direct view rendering displaysmay also be used.

In the indirect view configuration, the user wearing the headset 50 doesnot see the real-world environment through the display 56. Instead, inthe indirect view configuration, the display 56 shows real-time imageryof the real-world environment captured by the at least one camera, andAR objects are simultaneously rendered on the display 56 in locationsthat coincide with objects contained in the imagery of the real-worldenvironment captured by the at least one camera. In this manner, thewearer sees a view of the real-world environment indirectly, i.e., viacamera, simultaneously with an AR object, thereby providing an augmentedreality experience.

The details of the electronics and optics of the display system,including the AR object rendering, are not the subject of thisinvention, and any conventional or later-developed display and renderingsystem and/or technique may be used in implementations of the invention.Descriptions of various types of electronics and optics of an AR systemare provided in U.S. Patent Application Publication No. 2017/0053447,dated Feb. 23, 2017, U.S. Patent Application Publication No.2017/0109929, dated Apr. 20, 2017, U.S. Patent Application PublicationNo. 2017/0235143, dated Aug. 17, 2017, and U.S. Pat. No. 9,646,511issued May 9, 2017, each of which is incorporated by reference herein inits entirety.

In embodiments, the processor 52 and the memory 54 are embedded inand/or attached to the headset 50, e.g., as depicted in FIG. 1B.Alternatively, either or both of these elements may be contained in aseparate housing connected to the components of the headset 50 by wiredand/or wireless means. For example, the separate housing may be designedto be worn on a belt or to fit in the wearer's pocket, or one or more ofthese components may be housed in a separate computer 68 (e.g.,smartphone, tablet, laptop or desktop computer etc.) which communicateswirelessly with the display 56 and camera 58 in the headset 50 (e.g.,via interface 64), whereby the headset 50 and separate device constitutethe AR system.

FIGS. 3A, 3B, 4A, 4B, 5A, 5B, 6A, 6B, 7A, and 7B illustratefunctionality of the AR system in accordance with aspects of theinvention. These figures are shown with the goggle-like headset 50 ofFIG. 1A, but it is understood that the functionality described in thesefigures can be implemented in the glasses-like headset 50 of FIG. 1B andother shapes of headset 50.

In each of the pairs of figures (FIGS. 3A-B, FIGS. 4A-B, FIGS. 5A-B,FIGS. 6A-B, FIGS. 7A-B), the “A” figure depicts a real-worldenvironment, and the “B” figure depicts the corresponding view of whatthe user 40 sees via the display 56 of the headset 50. For example, FIG.3A depicts the real-world environment of a user 40 wearing the headset50 and looking in the direction of a patient 75 (also referred to as asubject). FIG. 3B shows the display 56 of the headset 50 correspondingto the real-world environment depicted in FIG. 3A. In a direct viewimplementation, FIG. 3B depicts the user 40 seeing the patient 75through the substantially transparent display 56. In an indirect viewimplementation, FIG. 3B depicts the display 56 showing a real-time imageof the patient 75 the, the image of the patient 75 being captured inreal-time by the one or more cameras 58 of the headset 50. In bothdirect view and indirect view implementations, if either one of the user40 and/or the patient 75 move relative to one another, the view through(or shown on) the display 56 will change in real-time, as is understoodin AR systems.

FIGS. 4A and 4B illustrate a continuation of the example from FIGS. 3Aand 3B. As shown in FIG. 4A, the user 40 manipulates a marker device 70to a location near/on the face of the patient 75. As shown in FIG. 4B,the user 40 sees (via the display 56) the movement and location of themarker device 70 relative to the patient 75 in real-time, e.g., the user40 continues to see the real-world environment in real-time via thedisplay 56.

Still referring to FIG. 4B, according to aspects of the invention, theuser 40 may provide an input to the system to generate a marker 80relative to the patient 75 as viewed via the display 56. The marker 80is a computer generated visual object that is overlaid onto thereal-world environment that is viewed via the display 56. Inembodiments, the marker 80 is displayed on the patient's face 75 at alocation that coincides with a distal tip of the marker device 70. Inthis manner, the user 40 may utilize the marker device 70 to selectivelyindicate locations (represented by markers 80) for injection therapy onthe patient 75. In this manner, the location of the marker 80 on thepatient 75 is defined by a location of a marker device 70 relative tothe patient 75 when the user input is received.

The user input to generate the marker 80 may take different forms. In afirst example, the marker device 70 may comprise a button that the user40 selects to provide the input. In this example, when the user 40presses the button, the marker device 70 sends a signal to the processor52 (via the interface 64), an application program/module being executedby the processor 52 generates the marker 80 based on the input, and theprocessor 52 causes the display 56 to visually show the marker 80superimposed on the real-world environment that is viewed via thedisplay 56.

In another example of the user input to generate the marker 80, thedistal end of the marker device 70 comprises a means to detect when themarker device 70 comes into physical contact with the patient 75. Themeans may comprise, for example, a depressible tip at the distal end ofthe marker device 70, the depressible tip being connected to anelectrical circuit that generates the input based on the tip beingdepressed when it contacts the patient. The means may comprise, inanother example, an electrical sensor at the distal end of the markerdevice 70, the sensor being configured to electrically detect physicalcontact with human skin, the sensor being connected to an electricalcircuit that generates the input based on the sensor detecting contactwith the patient. In these examples, in response to the means detectingthe marker device 70 comes into physical contact with the patient 75,the marker device 70 sends a signal to the processor 52 (via theinterface 64), an application program/module being executed by theprocessor 52 generates the marker 80 based on the signal, and theprocessor 52 causes the display 56 to visually show the marker 80superimposed on the real-world environment that is viewed via thedisplay 56.

In another example of user input to generate the marker 80, the systemmay be configured detect when the user 40 speaks a predefined word orphrase, and to generate the marker 80 based on such detecting. Forexample, the headset 50 or a connected computer device 68 may include amicrophone (not shown) that is configured to receive audio input, suchas words spoken by the user 40. An application program/module beingexecuted by the processor 52 may be programmed to analyze the audio datareceived at the microphone to detect predefined keywords or phrases inthe audio data. When one of the predefined keywords or phrases is spokenand detected in this manner, the application program/module beingexecuted by the processor 52 generates the marker 80 based on thedetecting, and the processor 52 causes the display 56 to visually showthe marker 80 superimposed on the real-world environment that is viewedvia the display 56.

In accordance with aspects of the invention, an applicationprogram/module executed by the processor 52 is configured to determine alocation of the distal end of the marker device 70 relative to the faceof the patient 75 for the purpose of determining a location of themarker 80 on the patient 75. This location determination may beperformed using data from the sensors 60 in the headset 50 and,optionally, using data from sensors in the marker device 70 that areconfigured to communicate with the sensors 60 in the headset 50. Forexample, one or more proximity sensors may be used to determine adistance from the headset 50 to the patient 75, and also a distance fromthe headset to the marker device 70. Image processing of the real-timeimagery obtained by the at least one camera 58 may be used to identify aposition of the marker device 70 relative to the face of the patient 75.Image processing of imagery obtained by the at least one camera 58 maybe used to generate a facial mapping of the patient 75. Combinations ofthis data may be used to determine a location on the face of the patient75 that corresponds to the marker 80 that is generated using the markerdevice 70 and user input. In embodiments, the system stores thedetermined location of a marker 80, e.g., in memory 54 or memory at thecomputer 68.

FIGS. 5A and 5B illustrate a continuation of the example from FIGS. 4Aand 4B. As shown in FIG. 5B, the user 40 has generated four markers 80on the face of the patient 75 (e.g., using the marker device 70 and userinput as described above). As shown in FIG. 5A, the marker device 70 isno longer near the face of the patient 75 (e.g., the user 40 has movedit away). As shown in FIG. 5B, the user 40 continues to see thereal-world environment in real-time via the display 56, with the addedimages of the four markers 80 superimposed on the face of the patient75.

As described herein, the system determines and saves a location on thepatient's face for each of the markers 80. In this manner, the systemcan be configured to constantly display the markers 80 on the samerespective locations of the face of the patient 75 even when one of theuser 40 and the patient 75 change positions relative to one another.

For example, as shown in FIGS. 6A and 6B, which are a continuation ofthe example from FIGS. 5A and 5B, the patient 75 and the user 40 havemoved relative to one another. Stated differently, the user 40 isviewing the patient from a first direction in FIG. 5A and from a seconddirection, different from the first direction, in FIG. 6A. As shown inFIG. 6B, the user 40 continues to see the real-world environment inreal-time via the display 56, e.g., the user 40 sees the patient fromthe second direction corresponding to the real-world environmentdepicted in FIG. 6A. According to aspects of the invention, the systemdisplays the markers 80 moving in real-time with the face of the patient(e.g., moving from that shown in FIG. 5B to that shown in FIG. 6B), suchthat the display 56 shows the markers 80 at the same locations of thepatient's face when the patient's face is moving due to relativemovement between the patient 75 and the user 40.

In embodiments, the application program/module executed by the processor52 is configured to perform continuous image processing of the real-timedata obtained by the at least one camera 58, and to continuously updatethe displayed positions of the markers 80 based on the image processingand the previously determined locations of the markers 80 relative tothe face of the patient 75. In a preferred embodiment, the system isconfigured to continuously update the displayed positions of the markers80 based on the image processing, the previously determined locations ofthe markers 80 relative to the face of the patient 75, and a determinedfacial mapping of the patient 75. In this manner, the user 40 may moverelative the patient 75 (e.g., as is common during treatment) andcontinue to see the markers 80 on the same locations of the patient'sface via the display 56 during such movement. In this manner, thedisplay is configured to continuously display the at least one marker 80at the location on the patient 75 as the real-world view of the subjectchanges.

FIGS. 7A and 7B illustrate a continuation of the example from FIGS. 6Aand 6B. As depicted in FIG. 7A, the user 40 wearing the headset 50 maymanipulate an injection device 85 (e.g., a syringe) to administerinjection therapy to the patient 75. As shown in FIG. 7B, the display 56of the headset 50 shows the real-world environment and the markers 80superimposed on the determined locations of the patient's face asdescribed herein. The user 40 may utilize the markers 80 as targetlocations for injections via the injection device 85. For example, asthe user 40 moves the injection device 85 relative the patient 75, theuser 40 can see via the display 56 when the injection device 85 (e.g., adistal end of a needle of a syringe) is at one of the markers 80.

According to aspects of the invention, the system stores data definingthe locations of the markers 80 relative to the face of the patient 75.The data may define the location of the marker 80 on the patient 75. Thedata may also define at least one of: a size of the marker, a shape ofthe marker; and a color of the marker. In an exemplary method of use,the markers 80 are generated during a first visit (e.g., at time t1),and stored in a patient profile. The patient returns at a later date fora second visit (e.g., at time t2 which may be days, weeks, or monthslater than t1). During the second visit, the system utilizes the storeddata in the patient profile to display the markers 80 via the display 56in the same locations on the patient's face as at the first visit. Inthis manner, the user 40 may utilize the system to perform repeatedinjections at same locations on the patient's face, even when theinjections are spaced apart by long periods of time between visits.

The markers 80 may have any desired shape, including but not limited to:dots, geometric shapes (e.g., circle, triangle, square, n-gon), lines,alphabet characters, and numeral characters. Different shaped markersmay be used on a same patient to indicate different aspects oftreatment. The system may be configured to permit the user to provideuser input (e.g., voice command, hand gesture, or computer 68) to selecta shape from plural predefined shapes for a marker 80.

The markers 80 may have any desired size. Different sized markers may beused on a same patient to indicate different aspects of treatment. Thesystem may be configured to permit the user to provide user input (e.g.,voice command or computer 68) to select a size from plural predefinedsizes for a marker 80.

The markers 80 may have any desired color. Different color markers maybe used on a same patient to indicate different aspects of treatment.The system may be configured to permit the user to provide user input(e.g., voice command or computer 68) to select a color from pluralpredefined color for a marker 80.

In embodiments, the system is configured to permit the user to provideinput to change one or more of the markers 80. For example, the systemmay be configured to show the markers 80 on an interface of the computer68, and permit the user to provide input (e.g., via touch screen, mouse,keyboard, etc.) to perform any one or more of the following changes to aselected marker 80: move the marker to another location in the patientface; delete the marker from the patient profile; change the shape ofthe marker; change the size of the marker, or change the color of themarker. In another example, the system may be configured to permit theuser to provide input to change/delete existing markers while the useris viewing the markers via the display 56. The user input in thisexample may be, for example, predefined voice commands, predefined handgestures, etc. that are recognized by the system and that are programmedto cause predefined changes to one of the markers 80. Any such changesto the existing markers are stored as data in the patient profile.

In embodiments, the system may be configured to determine a facial mapof the patient 75. The system may include hardware and be programmedwith software to perform the face mapping. For example, the system mayperform face mapping in a manner similar to the Kinect system byMicrosoft Corporation of Redmond, Wash., or the IphoneX system of AppleInc. of Cupertino, Calif. For example, as shown in FIG. 8, the headset50 may include a flood illuminator 90, infrared camera 95, front camera100, dot projector 105, proximity sensor 110, and ambient light sensor115. In embodiments, the dot projector 105 comprises an infrared emitterthat projects a number of dots (e.g., over 30,000 dots) in a predefinedpattern onto a person's face. The infrared camera 95 photographs theimage of the dots on the surface of the person's face. The processor 52runs an application program/module that is configured to determine asurface mapping of the person's face based on comparing the predefinedpattern of the projected dots to the image of the dots captured by theinfrared camera 95. The front camera 100 and the infrared camera 95 maycorresponds to the at least one camera 58 described with respect to FIG.2.

FIG. 9 shows an example of infrared dots projected onto a person's faceand a representation of a face map generated using the face mappingtechnique described with respect to FIG. 8. In this manner, the headset50 may be used to generate a 3-D face mapping of the patient 75, anddata defining the face map may be stored in the patient profile. Thedata defining the locations of the markers 80 may be defined in terms ofthe stored face map.

In embodiments, the system may be configured to automatically determinethe identity of a patient and access the patient profile, including theface map and defined locations of markers 80, based on the automaticidentification. For example, processor 52 may run an applicationprogram/module that is configured to identify a patient using facialrecognition techniques using the face map, other facial imageprocessing, retinal identification, etc. Based on identifying thepatient, the system may be configured to access the patient profile forthe identified patient and automatically display (via the display 56)this patient's markers 80 superimposed on the patient's face.Additionally, or alternatively, the system may be configured to permitthe user to provide user input (e.g., voice command or computer 68) toselect input the identity of the patient and, based on this input, toaccess the patient profile for the identified patient and automaticallydisplay the markers 80 (via the display 56) that correspond to thispatient.

FIG. 10 shows an exemplary marker device 70 in accordance with aspectsof the invention. In embodiments, the marker device 70 comprises astylus. As described herein, the marker device 70 may include a button90 that is operatively connected to circuitry within the device thatgenerates a user input signal when the button is pressed. As describedherein, the marker device 70 may include a means 95 to detect when themarker device 70 comes into physical contact with the patient 75. Themeans 95 may comprise, for example, a depressible tip at the distal endof the marker device 70, the depressible tip being connected to anelectrical circuit that generates the input based on the tip beingdepressed when it contacts the patient. The means 95 may comprise, inanother example, an electrical sensor at the distal end of the markerdevice 70, the sensor being configured to electrically detect physicalcontact with human skin, the sensor being connected to an electricalcircuit that generates the input based on the sensor detecting contactwith the patient. The marker device 70 may communicate with theinterface 64 via wired or wireless connection.

FIG. 11 shows a flowchart of a method in accordance with aspects of theinvention. The steps of FIG. 11 may be implemented using an AR system asdescribed herein, for example, and are described using reference numbersof elements depicted in FIGS. 1-10. The flowchart illustrates thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention.

At step 201, the AR system generates a face map of a patient. The facemap may be generated in the manner described with respect to FIGS. 8 and9, or using other face mapping techniques. Data defining the face mapmay be stored in a patient profile for the patient being mapped.

At step 202, the AR system generates one or more markers 80, e.g., inthe manner described herein with respect to FIGS. 4A and 4B. At step203, the display 56 of the AR system displays the markers 80 overlaid onthe patient, e.g., as described with respect to FIGS. 5A, 5B, 6A, 6B. Atstep 204, the user 40 wearing the headset 50 of the AR system providesinjection therapy to the patient while viewing the patient 75 and themarkers 80 via the display 56, e.g., as described with respect to FIGS.7A and 7B.

In an exemplary method, steps 201-204 may be performed at a first visit,and steps 205-208 may be performed at a second visit. At step 205, thepatient returns for the second visit and the AR system automaticallyidentifies the patient. In embodiments, the automatic identification isperformed using the face map from step 201, e.g., in the mannerdescribed herein, or by using other camera and computer-based automaticrecognition techniques.

At step 206, the display 56 of the AR system displays the markers 80overlaid on the patient, e.g., as described with respect to FIGS. 5A,5B, 6A, 6B. In embodiments, after recognizing the patient at step 205,the AR system accesses the patient profile, obtains data defining themarkers 80 for this patient, and displays the markers 80 at the definedlocations on the patient's face as seen via the display 56.

At step 207, the user may optionally modify one or more of the existingmarkers 80 and/or may add at least one new marker 80. For example, asdescribed herein, the user may provide input to change thesize/shape/color/location of an existing marker 80 that is displayed bythe display 56. Additionally, or alternatively, the user may provideuser input to delete an existing marker 80 that is displayed by thedisplay 56. Additionally, or alternatively, the user may provide userinput to generate a new marker 80, e.g., in the manner similar to step202.

At step 208, the user 40 wearing the headset 50 of the AR systemprovides injection therapy to the patient while viewing the patient 75and the markers 80 via the display 56, e.g., as described with respectto FIGS. 7A and 7B.

Implementations of the invention may be used to provide accuracy andrepeatability in administering injections to a subject. Aspects may beused to provide targeted treatment of: brow lines, glabella furrows,crows feet, nasojugal folds, nasolabial folds, marionette lines, chinline, and platysmal bands. Implementations of the invention are notlimited to use with the face of a subject, and instead may be used onany part of a subject. Aspects described herein permit the provider toreproduce injections over multiple visits and/or alter injection habitsin order to maximize the subject results.

Implementations of the invention may be used with different types ofinjection therapy. For example, aspects may be used to provide targetedinjection therapy with: neurotoxins, subcutaneous dermal enhancers,insulin, antibiotics, hormones, chemotherapeutic agents,anti-inflammatory agents, other biological agents, Botox, subdermalfiller such as Restylane, Juvederm, etc.

An aspect of the invention includes an AR system comprising the headset50 programmed with logic (e.g., an application program/module) thatcauses the headset to perform one or more of the functions describedherein. Another aspect of the invention includes a computer programproduct, provided as a physical storage medium or as a download, that isconfigured to be executed by an AR headset (or by a computer deviceconnected to an AR headset) to cause the AR headset to perform one ormore of the functions described herein. Another aspect of the inventionincludes an AR system comprising both the marker device 70 and theheadset 50 programmed with logic (e.g., an application program/module)that causes the headset to perform one or more of the functionsdescribed herein. Another aspect of the invention includes a kitcomprising the marker device 70 and a computer program product, providedas a physical storage medium or as a download, that is configured to beexecuted by an AR headset (or by a computer device connected to an ARheadset) to cause the AR headset to perform one or more of the functionsdescribed herein. Another aspect of the invention includes a method ofproviding targeted injection therapy as described herein. The computerprogram product may comprise a computer readable storage medium havingprogram instructions embodied therewith, the program instructionsexecutable by a computing device to cause the computing device toperform one or more functions described herein.

It is noted that the foregoing examples have been provided merely forthe purpose of explanation and are in no way to be construed as limitingof the present invention. While the present invention has been describedwith reference to an exemplary embodiment, it is understood that thewords which have been used herein are words of description andillustration, rather than words of limitation. Changes may be made,within the purview of the appended claims, as presently stated and asamended, without departing from the scope and spirit of the presentinvention in its aspects. Although the present invention has beendescribed herein with reference to particular means, materials andembodiments, the present invention is not intended to be limited to theparticulars disclosed herein; rather, the present invention extends toall functionally equivalent structures, methods and uses, such as arewithin the scope of the appended claims.

What is claimed is:
 1. A augmented reality (AR) system for administeringinjections, comprising: an AR headset comprising a display that isconfigured to display at least one marker superimposed on a real-worldview of a subject, the at least one marker being displayed at auser-defined location on the subject.
 2. The system of claim 1, whereinthe display is configured to continuously display the at least onemarker at the location on the subject as the real-world view of thesubject changes.
 3. The system of claim 1, wherein the marker isgenerated based on user input received from a user wearing the ARheadset.
 4. The system of claim 3, wherein the location on the subjectis defined by a location of a marker device relative to the subject whenthe user input is received.
 5. The system of claim 4, wherein the userinput is a voice command.
 6. The system of claim 4, wherein the userinput is a button press on the marker device.
 7. The system of claim 4,wherein the user input is based on a means to detect when the markerdevice comes into physical contact with the subject.
 8. The system ofclaim 1, wherein data defining the at least one marker is stored incomputer memory.
 9. The system of claim 8, wherein the data defines thelocation on the subject.
 10. The system of claim 8, wherein the datadefines at least one of: a size of the at least one marker, a shape ofthe at least one marker; and a color of the at least one marker.
 11. Thesystem of claim 8, wherein the system is configured to receive userinput to change at least one of: the location on the subject; a size ofthe at least one marker, a shape of the at least one marker; and a colorof the at least one marker.
 12. The system of claim 8, wherein thesystem is configured to: automatically identify the subject; in responseto the automatically identifying the subject, access the data definingthe at least one marker; and in response to the accessing the data,automatically display the at least one marker superimposed on anotherreal-world view of a subject, the at least one marker being displayed atthe user-defined location on the subject.
 13. The system of claim 1,wherein: the AR headset comprises a direct view device; the display issubstantially transparent; and the real-world view of the subject isseen through the substantially transparent display.
 14. The system ofclaim 1, wherein: the AR headset comprises an indirect view device; andthe real-world view of the subject is real-time camera imagery that isdisplayed via the display.
 15. The system of claim 1, wherein the ARheadset comprises a face mapping system.
 16. The system of claim 15,wherein the face mapping system comprises: an infrared emitter thatprojects a number of dots in a predefined pattern onto a surface of aperson's face; and an infrared camera that photographs an image of thedots on the surface of the person's face.
 17. The system of claim 15,wherein the location of the at least one marker on the subject isdefined relative to a facial map of the subject generated by the facemapping system.
 18. The system of claim 1, wherein the AR headsetcomprises one of: goggles, glasses, a headband, and a helmet.
 19. Amethod of using augmented reality (AR) for administering injections,comprising: receiving user input defining at least one marker at alocation on a subject; displaying, by a display of an AR headset, the atleast one marker superimposed on a real-world view of the subject, theat least one marker being displayed at the user-defined location on thesubject.
 20. The method of claim 19, further comprising continuouslydisplaying the at least one marker at the location on the subject as thereal-world view of the subject changes.